Virtual reality presentation of real world space

ABSTRACT

Methods and systems are provided for delivering a virtual reality (VR) experience of a real world space to a remote user via a head mounted display (HMD). A method provides for sending a request for the VR experience of the real world space and identifying a seat selection made by the user. The method includes operations for mapping the seat selection to a real world capture system for capturing video and audio at a location that corresponds to the seat selection and receiving real world coordinates for the real world capture system. Further, the method accesses a user profile of the user and receives a video stream of the real world space captured by the real world capture system. The method is able to identify and reskin a real world object with a graphical content element by overlaying the graphical content item in place of the image data associated with the real world object.

CLAIM OF PRIORITY

This application is a non-provisional application of U.S. ProvisionalApplication No. 62/566,266, filed Sep. 29, 2017, entitled “VIRTUALREALITY PRESENTATION OF REAL WORLD SPACE,” which is herein incorporatedby reference.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to virtual reality (VR)presentations of real world spaces and more particularly to methods andsystems for providing a user of a head mounted display (HMD) with a VRexperience of a live event such as an e-sports event that is remote tothe user.

BACKGROUND

Electronic sports, or “esports,” are becoming an increasingly popularform of entertainment for spectators. Esports are a form of competitiontypically involving multiplayer video games that are facilitated byelectronic systems such as video game servers and computer systems. Someof the more popular video game genres that are played in an esportscontext are real-time strategy games, combat games, first-person shooter(FPS) shooter games, and multiplayer online battle arena (MOBA) games.Esports events become spectator events when the Esports game is playedlive in front of an audience. Many esports events are held in largearenas and provide audience members a view of the game action on a largedisplay such as a jumbotron as well as live commentary from acommentator team. Thus, an esports live audience member is given animmersive real world experience of the game action, the players, thecommentators, and the audience itself.

In many instances, esports fans that are not able to attend an esportsevent in person are still able to tune in via an online live stream ofthe event. Thus, remote spectators are able to get a generic videostream of the event that is produced by, for example, the organizationhosting the event or a distribution company distributing the event. Ineither scenario, current video streams of esports events are notimmersive. For example, a remote spectator watching an esports videostream on a display will not feel as if they are immersed in the arenawhere the esports event is being hosted. Moreover, there may be aplurality of camera views and “screen views” (e.g., what a player'sscreen is displaying) that may be of interest to the spectator. Currentbroadcasts of esports to not allow consumers to choose which of theviews they want shown.

It is in this context that embodiments arise.

SUMMARY

Embodiments of the present disclosure relate to methods and systems fordelivering a virtual reality (VR) presentation of a real world space toa remote user via a head mounted display (HMD). In one embodiment, amethod includes an operation for sending a request for the VRpresentation of the real world space and for identifying a seatselection within the real world space made by the user, the seatselection being mapped to a real world capture system configured forcapturing video and audio at a location that corresponds to the seatselection. The method also includes operations for receiving mappingdata for the real world capture system, the mapping data including realworld coordinates for the real world capture system. The method is ableto access a user profile of the user having user preferences. Accordingto some embodiments, the method receives a video stream of the realworld space from the real world capture system, the video streamincluding a plurality of video images captured by one or more cameras ofthe real world capture system, the plurality of images being presentedin the HMD from a perspective associated with the real world coordinatesof the real world capture system corresponding to the seat selection.The method further includes operations for identifying a real worldobject configured for reskinning during presentation from the pluralityof images of the video stream. According to this and other embodiments,the method reskins the real world object within the plurality of videoimages based on the user profile, the reskinning including overlaying agraphical content element in place of the image data associated with thereal world object. In certain embodiments, the overlaying isgeometrically adjusted based on the perspective associated with the realworld capture system corresponding to the seat selection. The methodthen presents the video stream having been reskinned to the user via theHMD for the VR presentation.

Other aspects of the disclosure will become apparent from the followingdetailed description, taken in conjunction with the accompanyingdrawings, illustrating by way of example the principles of thedisclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure may best be understood by reference to the followingdescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 is a conceptual illustration of an embodiment for delivering avirtual reality (VR) presentation of an esports events taking place in aremote real world space.

FIG. 2 shows an overall flow of a method for reskinning a VRpresentation of a real world space according to one embodiment.

FIG. 3 shows an overall flow of a method for using data gathered fromthe real world space to provide a VR presentation of a real world spacehaving a point of view corresponding to a seat selection made by a user,according to one embodiment.

FIG. 4 shows an overall flow for reskinning a static real world objectof a real world space during VR presentation of the real world space,according to one embodiment.

FIG. 5 shows an overall flow for reskinning a moving real world objectof the real world space during VR presentation of the real world space,according to one embodiment.

FIG. 6 shows a conceptual illustration of a remote spectator making aseat selection for viewing a live event via an HMD, according to oneembodiment.

FIG. 7 shows a conceptual illustration of a process that maps a remotespectator's seat selection to a real world capture system, the realworld capture system including a plurality of cameras and microphonesfor capturing events in the real world, according to one embodiment.

FIG. 8 shows a conceptual illustration of obtaining point of viewmapping data associated with the real world capture system from realworld space data and real world environment data, according to oneembodiment.

FIG. 9 shows a conceptual illustration of reskinning real world objectsin a real world view of an esports event.

FIGS. 10A and 10B show conceptual illustrations of geometricallyadjusting a graphical element for overlaying on top of an ad forreskinning based on the perspective of the real world capture system.

FIG. 11 shows additional embodiments of real world objects that areconfigurable to be reskinned.

FIG. 12 shows a conceptual illustration of a remote spectator beingpresented with an in-game fly-through view of the video game beingplayed at an esports event, according to one embodiment.

FIG. 13 shows a conceptual illustration of a remote spectator choosingbetween different views of an esports event for display, according toone embodiment.

FIG. 14 shows a diagram of an exemplary computing system that enables VRpresentation of an esports event via HMD to a user, according to oneembodiment.

FIG. 15 shows a conceptual illustration of two remote spectatorsinteracting with one another virtually within the context of an esportsevent, according to one embodiment.

FIG. 16 shows an embodiment of a head mounted display (HMD) capable ofbeing used with methods and processes presented here.

DETAILED DESCRIPTION

The following embodiments describe methods, computer programs, andapparatuses for providing an immersive virtual experience of a liveevent to a remote spectator via HMD. It will be obvious, however, to oneskilled in the art, that the present disclosure may be practiced withoutsome or all of these specific details. In other instances, well knownprocess operations have not been described in detail in order to notunnecessarily obscure the present disclosure.

As electronic sports, or “esports,” are becoming an increasingly popularform of entertainment for spectators. Esports are a form of competitiontypically involving multiplayer video games that are facilitated byelectronic systems such as video game servers and computer systems. Someof the more popular video game genres that are played in an esportscontext are real-time strategy games, combat games, first-person shooter(FPS) shooter games, and multiplayer online battle arena (MOBA) games.Esports events become spectator events when the Esports game is playedlive in front of an audience. Many esports events are held in largearenas and provide audience members a view of the game action on a largedisplay such as a jumbotron as well as live commentary from acommentator team. Thus, an esports live audience member is given animmersive real world experience of the game action, the players, thecommentators, and the audience itself.

In many instances, esports fans that are not able to attend an esportsevent in person are still able to tune in via an online live stream ofthe event. Thus, remote spectators are able to get a generic videostream of the event that is produced by, for example, the organizationhosting the event or a distribution company distributing the event. Ineither scenario, current video streams of esports events are notimmersive. For example, a remote spectator watching an esports videostream on a display will not feel as if they are immersed in the arenawhere the esports event is being hosted. Moreover, there may be aplurality of camera views and “screen views” (e.g., what a player'sscreen is displaying) that may be of interest to the spectator. Currentbroadcasts of esports to not allow consumers to choose which of theviews they want shown.

Methods and systems presented here allow a remote user of an HMD to viewa VR presentation of a real world event such as an esports event.According to some embodiments, real world data such as video, audio, andother data are captured by devices located within a real world spacesuch as an auditorium, an arena, a concert hall, a warehouse. The realworld data including video and audio data are delivered to a VR hostingserver or other content provider for processing. The VR hosting serveris able to process the real world data and render a VR presentation ofthe real world event. The VR presentation data is sent over a network toan HMD of the user or to a computing device connected to the HMD.

According to some embodiments, certain areas or regions, or generally,real world objects, may be visually modified for the VR presentation. Inone embodiment, the visual modification includes reskinning real worldobjects such as advertisements. Reskinning includes processes thatoverlay new content on top of existing content or replace or integratepixel data associated with new content in place of the pixel dataassociated with existing content within the video frames. Embodimentscontemplated here may use either method of reskinning. As a result,remote spectators of real world events may be provided with reskinnedcontent that is more attuned to their interests and free of content thatis unwanted.

FIG. 1 is a conceptual illustration of an embodiment for delivering avirtual reality (VR) presentation 106 of an esports event 100 to aremote user 101 via HMD 102 and VR environment 104. The esports event100 is shown to include a first team 112 and a second team 114 playingagainst each other in a multiplayer video game hosted by a local server116. Team banners 120 and 122 are shown indicate which team is which. Areal-time presentation 103 of the happenings of the video game isdisplayed on a main display 118 such as a jumbotron, and includes livevideo of the video game as it is being played from various points ofview.

Each player in the esports event 100 is shown to have their own gamingstation where they can interact with a monitor and input devices whileplaying the multiplayer game. Each gaming station also has anaudience-facing panel that is shown to display a live portrait view ofeach player as they play the multiplayer game. Also shown in FIG. 1 area pair of commentators 105 who may provide play-by-play and colorcommentary regarding the game and its players.

The esports event 100 is further shown to include a plurality ofphysical advertisements that are viewable by audience 132. The physicaladvertisements may include printed banner advertisements, projectedadvertisements, LED displayed advertisements and the like. For example,the esports event is shown to include a left-wall ad 124 for milk, aright-wall ad 126 for instant noodles, a stage-wall ad 130 for energydrinks, and a commentator desk ad 128 for gaming products.

The audience 132 is shown to include a plurality of seats facing thestage of the esports event 100. According to some embodiments, a remoteuser 101 is able to choose a particular seat of the plurality of seatsfor the VR presentation 106 of the esports event 100. In the embodimentshown, the user 101 has chosen seat 134 having a point of view 136 fromwhich to view the esports event 100 via the VR presentation 106. As willbe described in more detail below, the point of view 136 of seat 134 isprovided by a real world capture system 138 having a plurality ofcameras and microphones located at seat 134 to capture the esports event100 from the point of view 136 of the seat 134.

HMD 102 is shown to have access to the esports event 100 through a VRhosting server 107. The VR hosting server 107 receives real world dataand game state data provided by the local server 116 for processing a VRpresentation. For example, video game data such as game stateinformation of the video game, as well as the video scenes of the videogame being rendered from different points of view may be provided to VRhosting server 107 for processing. Moreover, real world data captured bythe real world capture system 136 may be routed by local server 116 tothe VR hosting server 107 for processing. In some embodiments, there maybe plurality of real world capture systems for capturing real world datafrom a plurality of vantage points within esports event 100. As aresult, a different server or plurality of servers may be used forserving real world data captured by the plurality of real world capturesystems to remote users that is distinct from local server 116 forhosting the multiplayer video game.

The VR presentation 106 of the esports event 100 is shown to be from apoint of view or vantage point of the seat 134 selected by remote user101. As a result, the VR presentation 106 may provide a visual and audioexperience of the esports event 100 as if remote user 101 were locatedat seat 134. In some embodiments, the real world capture system 138 isable to provide a 360 degree view of the esports event 100 and thephysical space surrounding it such that the remote user 101 is also ableto have a 360 degree view of the esports event 100 by moving turningtheir head or body. As a result, the remote user 101 is provided with animmersive VR presentation 106 of the esports event 100 that currentesports broadcasts are unable to provide because the remote user 101 isprovided with an experience as if they were inside the arena hosting theesports event 100.

According to the embodiment shown in FIG. 1, a number of modificationshave been made to the VR presentation 106 that differs from the realworld data captured by the real world capture system 138. For example,the left-wall ad for milk 124 has been reskinned to be a left-wall ad124′ for PlayStation in the VR presentation. Likewise, the right-wall ad126 for instant noodles in the real world space appears to remote user101 as a right-wall ad 126′ for Farpoint. Moreover, the stage-wall ad136 and the commentator desk ad 128 have been reskinned to display adcontent that is different than that of the real world space. Each of thereskinned ads 124′-130′ is shown to keep their perspective, size, andgeometric proportions in the VR presentation 106 relative to that in thereal world with respect to the point of view of the seat 134 and thereal world capture system 138. For example, left-wall ad 124′ appears tothe user 101 with the same perspective as left-wall ad 124 would if theVR presentation 106 were to occur without reskinning.

As mentioned earlier, reskinning may involve overlaying replacementcontent on top of existing content by inserting frames into a videostream having the existing content such that the new content isdisplayed instead of the existing content. In other embodiments,reskinning may involve replacing, blending, or integrating new pixeldata associated with the new content into the existing pixel dataassociated with the existing content.

Moreover, it is envisioned that reskinning existing content with newcontent involves analyzing a perspective of the existing content suchthat the new content is reskinned having the same or similarperspective. In some embodiments, a method or system may gather dataregarding the real world coordinates of each of the vertices of theexisting content as well as real world coordinates of the real worldcapture system (e.g., the camera capturing the existing content). Inthese embodiments, a perspective of the existing content may bedetermined based on the gathered coordinates. For example, the verticesof the new content may be placed virtually in 3-dimensional space tomatch the coordinates of the vertices of the existing content andprojected back into 2-dimensional space. The resulting 2-dimensionalprojection of the new content will have the same or a similarperspective as the existing content. In other embodiments of reskinning,reskinning may occur in 2-dimensional space by distort transforming orperspective distorting new content to occupy the same 2-dimensional areaas the existing content within a video image.

Moreover, it is contemplated that reskinning used with the methods andsystems presented here are enabled to dynamically reskin existingcontent that moves in relation to a camera point of view. For example,an object moving in the real world may be reskinned by tracking theobject in each of the video frames and performing the reskinning on theobject within the video frames. As the object moves, a perspective ofthe object may change as well. The reskinning methods contemplated hereare able to track these changes in real time and reskin new contentaccording to the changing perspective.

In other embodiments, the real world capture system may move within thereal world space. Thus, even if real world objects are stationary in thereal world, a perspective of a real world object will change while thecamera moves. Methods and systems contemplated here are able to reskinreal world objects according to a changing perspective as a camera of areal world capture system moves about the real world. Similar toreskinning a moving object, reskinning a stationary object for a movingcamera point of view involves tracking an object over a series of videoframes and overlaying or replacing existing image data associated withexisting content with new image data associated with new content.

In some embodiments, existing content (e.g., real world ads) may beautomatically identified using classifiers, object recognition,character recognition, and artificial intelligence. In theseembodiments, existing content automatically identified are tagged to bereplaced by a reskinning logic. For example, the VR hosting service maybe instructed to identify and reskin a particular brand of beef jerkyautomatically ever time indicia of that brand is captured. It isenvisioned that the VR hosting service may be enabled to automaticallyreskin indicia of the brand of beef jerky for the eventual VRpresentation.

In some embodiments, the content used for reskinning (e.g.,“PlayStation,” FarPoint,” etc.) may be based upon user preferences, usersettings, user demographic, user browsing history, user shoppinghistory, user viewing history, and the like. As a result, the remoteuser 101 may be provided with advertisement content that matches theinterest of the remote user 101 in a way that does not distract from VRpresentation 106 of the esports event 100.

FIG. 2 shows an overall flow of a method for reskinning a VRpresentation of a real world space according to one embodiment. Themethod includes an operation 210 to request entry to a real world spacefrom a user at a remote location to experience a VR presentation of thereal world space using an HMD. The method then flows to operation 220,which serves to identify a user profile of the user, the user profileincluding user preferences. In some embodiments, the use profile and theuser preferences may include data on user interest, user demographic,user shopping history, user browsing history, user viewing history, aswell as social network data on social friend activity, interests, andthe like. As a result, the method may be enabled to learn of products,services, events, etc., that are of interest to the remote user.

In operation 230, the method receives a seat selection from the user. Aswill be discussed in more detail below, the user may be offered aplurality of seats that correspond to real world locations in the realworld space for viewing real world action. Different seats may providedifferent VR presentations due to the different seats having a differentpoint of view or vantage point of the real world action. In someembodiments, different seats may be priced at different values. Forexample, the best seat in the house that is front and center of thestage may have a higher price or requirement than a seat that is locatedfarther away from the stage.

In operation 240, the method requests a live video stream having avantage point that corresponds to the seat selection of the remote user.As shown in FIG. 1, the live video stream may be provided by a realworld capture system having a plurality of cameras and microphones. Themethod then flows to operation 250, which serves to receive real worldcoordinate location for the vantage point. Thus, for example, operation250 may obtain real world coordinate location of the real world capturesystem shown in FIG. 1. Additionally, the method is able to obtain realworld mapping data of the real world space, which may include real worldcoordinates of objects within the real world space. For example, if thereal world space is an arena having a stage, the real world mapping datamay provide information on where the stage is relative to the vantagepoint.

The method then flows to operation 260, which serves to execute adynamic reskinning process for real world objects viewed on the HMDbased on the real world coordinate location for the vantage point, thereal world mapping data, and the user preferences. The real worldobjects used for dynamic reskinning may include any object in the realworld, such as advertisements, banners, displays, surfaces, patches,floors, ceilings, electronic devices, etc.

FIG. 3 shows an overall flow of a method for using data gathered fromthe real world space to provide a VR presentation of a real world spacehaving a point of view corresponding to a seat selection made by a user,according to one embodiment. Operation 310 identifies a seat selectionmade by the user for experiencing a VR presentation of a real worldspace. The method then flows to operation 320, which serves to obtainreal world coordinate location for the seat selection for processing theVR presentation. The process for obtaining real world coordinates forthe seat selection may involve mapping the seat selection to a realworld capture system located in the real world that corresponds to theseat selection.

In some embodiments, a seat selecting process is similar to selectingseats for live events where a ticket buyer is provided with an option tobuy tickets for particular seats within an arena. Thus, each seatselection maps to a real world seat in the real world space. In someembodiments, for every seat selection, there is a distinct real worldcapture system that captures events in the real world from a locationthat corresponds to the seat selection. In these embodiments, the realworld coordinates for the real world seat will be similar to the realworld coordinates of the real world capture system. In otherembodiments, there may be two or more seat selections that map to thesame real world capture system. As a result, the method will obtain thereal world coordinates of the real world capture system for purposes ofprocessing the VR presentation. In other embodiments, there may be acombination of one-to-one mapping and many-to-one mapping of seatselections to real world capture systems. Operation 320 is able to mapthe seat selection to a real world capture system that is the best fitin terms of proximity, viewing angle, etc. relative to the seatselection.

If the seat selection is for a seat in an auditorium or arena forhosting an esports event, the real world coordinate location of the realworld capture system obtained by operation 320 help to identify wherethe real world capture system is relative to objects in the auditoriumor arena. For example, a seat selection of AA01 may be mapped to a realworld capture system that has a real world coordinate location of (x, y,z) within a coordinate system of the auditorium or arena.

Operation 330 is configured to access space mapping data for the realworld space. Space mapping data may include a virtual map of thephysical space in terms of the coordinate system of the auditorium orarena mentioned above. The space mapping data generally helps identifylocations of objects, walls, floors, ceilings, seats, stages, etc.Again, if the real world space is an auditorium or arena hosting anesports event, the space mapping data may include coordinate data of theesports players, the commentators, the main display, the stage,advertisements, and other points of interest.

In operation 340, the method accesses space environment data for thereal world space from the real world capture system that the seatselections maps to. Space environment data may include any type of datacapable of being captured by the real world capture system. For example,as mentioned above, a real world capture system may include an array ofcameras, microphones, and other sensors for measuring distances ofobjects. As a result, space environment data may include data gatheredfrom a sound calibration process that detects acoustics properties ofthe auditorium relative to the position of the real world capturesystem. The acoustic properties detected may include directionalacoustic reflectance, impedance, gain, harmonics, background or ambientnoise, etc.

The space environment data obtained by operation 340 may also includemeasurements of distances between the real world capture system andvarious objects within the real world space. Angular distances betweenrays traced to vertices of real world objects such as corners of a largeadvertisement within the real world space may also be measured. Forexample, the space mapping data may be able to determine coordinatelocation of each of the four vertices of a large rectangularadvertisement in the real world space. As a result, the spaceenvironment data may be usable to construct a 3-D model real worldobjects in the real world space.

In operation 340, data gathered from operation 320 (e.g., real worldcoordinate location for the real world capture system), operation 330(e.g., space mapping data), and operation 340 (e.g., space environmentdata) are used to process point of view mapping data. Point of viewmapping data may include data that is relevant to processing a VRpresentation of the real world space as well as reskinning of real worldobject for the VR presentation. For example, point of view mapping datamay include distances measured to real world objects, angular distancesof rays traced to vertices of real world objects, real world coordinatesof real world objects (e.g., coordinates of real world objectsindependent of the real world capture system), camera coordinates ofreal world objects (e.g., coordinates of real world objects relative tothe real world capture system). In addition, the point of view mappingdata may also include data on the acoustic properties of the real worldspace at the location of the real world capture system.

The method shown in FIG. 3 then flows to operation 360, which providesthe VR presentation of the real world space to the user via an HMDaccording to the point of view mapping data processed by operation 350.The VR presentation may include live audio and video streams captured bythe real world capture system. In certain embodiments, the VRpresentation may include modified versions of the live audio and videostreams according to the point of view mapping data. An example ofmodifying the audio stream includes mixing various audio inputs suchaudio captured at the real world capture system with audio associatedwith the audience, players, commentators, game sounds, etc. An exampleof modifying the video stream includes reskinning a real world objectsuch as a large rectangular ad to appear as a large rectangular ad forsomething else in the VR presentation.

FIG. 4 shows an overall flow for reskinning a static real world objectof a real world space during VR presentation of the real world space,according to one embodiment. The method begins with operation 410 forrequesting a VR presentation of a real world space from a remote uservia HMD. The method then flows to operation 420 for receiving, from areal world capture system, a live video stream of the real world space,the video stream including a plurality of video images. If the realworld space is an arena hosting an esports event, the video stream mayinclude video of the players of each team, a main display of the videogame, other spectators, commentators, advertisements, etc. As mentionedabove, the video feed may include a 360 degree horizontal view of thereal world space that allows an HMD user receiving the video feed tolook around in 360 degrees by turning their head or body. In otherembodiments, a 360 degree vertical view of the real world space is alsoprovided.

The method then flows to operation 430, which serves to identify realworld coordinate location of the real world capture system, the realworld coordinate location usable to determine a perspective for the realworld capture system. For example, by having the real world coordinatesof the real world capture system, the method is able to analyze theperspective associated with various real world objects or scenescaptured by the real world capture system. In one example, if real worldcoordinate are known for the vertices of a large rectangularadvertisement in the real world space, then the method may use the realworld coordinate (e.g., x, y, z) of the real world capture system todetermine a perspective associated with the advertisement when capturedby the real world capture system.

In operation 440, the method accesses a user profile to identifypreferences such a location, browsing history, purchase history, searchhistory, demographic and interest. In operation 450, the methodidentifies, from the plurality of video images of the live video streamof the real world arena, a stationary real world object configurable tobe reskinned. For example, operation 450 may identify that a largerectangular advertisement is configurable to be reskinned with someother content. There are any number of types of static real worldobjects and portions of real world objects that may be configurable tobe reskinned. For example, portions of walls, curtains, displays, gamingstations, portions of the stage, etc. may all be configurable to bereskinned as static real world objects. It will be appreciated that areal world object that is configurable to be reskinned is also taken tomean portions of real world object that are configurable to bereskinned.

The method then flows to operation 460, which serves to reskin, based onthe user profile of the user and based on the real world coordinatesystem of the real world capture system, the stationary real worldobject within the live video stream. In the embodiment shown, thereskinning includes overlaying graphical content in place of the imagedata associated with the real world object in the live video stream.Moreover, it is shown that the overlaying is to be geometricallyadjusted based on the perspective associated with the real worldcoordinate location of the real world capture system.

For example, if a large rectangular ad is identified by operation 450 asa stationary real world object configurable to be reskinned, thenoperation 460 would be operable to identify a graphical content elementthat is of interest to the user for reskinning, for example, an ad forsneakers. Operation 460 is able to overlay the ad for sneakers in placeof the image data associated with the large rectangular ad, for example,by creating new video stream having the ad for sneakers in place of thepixel values that were previously associated with the large rectangularad. Moreover, it is contemplated that the ad for sneakers is to beadjusted geometrically based on the perspective of the real worldcapture system. For example, if the large rectangular ad appears askewfrom the perspective of the real world capture system, the replacementgraphical content of the ad for sneakers will also be made to appearskew in a similar way as the large rectangular ad. Operation 470 thenpresents the modified video stream having the real world objects beingreskinned.

FIG. 5 shows an overall flow for reskinning a moving real world objectof the real world space during VR presentation of the real world space,according to one embodiment. The method proceeds similar to the methodshown in FIG. 4. For example, operations 510 and 520 are configured torespectively request a VR presentation of a real world space from aremote user presented via HMD and to receive a live video stream of thereal world space from a real world capture system.

In operation 530, the method identifies a real world coordinate locationof the real world capture system for determining a perspective of thereal world capture system. In operation 540, the method accesses a userprofile of the user to identify user preferences such as location,browsing history, purchase history, search history, demographics,interests, etc. In operation 550, the method identifies a dynamic objectconfigurable to be reskinned from a plurality of video images of thelive video stream of the real world space.

The method of FIG. 5 then flows to operation 560, which serves to trackthe dynamic object within the plurality of video frames of the livevideo stream. A dynamic real world object that is configurable to bereskinned may include attire that players or spectators are wearing,furnishings, and other gear that tend to move within the physical space.Operation 560 is configured to track their movements for reskinning inlater operations.

Operation 570 of the method allows for the reskinning of the dynamicreal world object within the video stream based on the user profile ofthe user and based on the real world coordinate location of the realworld capture system. According to the embodiment shown, the reskinningincludes overlaying replacement graphical content in place of image dataassociated with the real world object, wherein the overlaying isgeometrically adjusted based on the perspective of the real worldcoordinate location of the real world capture system. In operation 580,the method presents the reskinned video stream to the user via the HMD.

FIG. 6 shows a conceptual illustration of a remote spectator 600 makinga seat selection for viewing a live event via an HMD 601, according toone embodiment. The remote spectator 600 is shown to be provided with aseat selection interface 602 that is viewable with the HMD. The seatselection interface 602 is able to convey auditorium and seatinformation to the user. In some embodiments, the seat selectioninterface 602 is also able to convey price information of each seat fora VR presentation of a real world event.

According to FIG. 6, the remote spectator 600 is able to preview variousseats shown in the seat selection interface 602. For example, the remotespectator 600 is shown to make a first selection 604 for a seat that istoward the front of the auditorium. The seat selection interface 602 maythen provide a first preview 606 that includes a VR scene of theauditorium from the seat corresponding to the first selection 604.Likewise, when remote spectator 600 makes a second selection 608 that isin the balcony of the auditorium, the seat selection interface 602 maythen provide a second preview 610 including a VR scene of the auditoriumfrom the seat corresponding to the second selection 618. It iscontemplated that for some embodiments the first and second previews 606and 610 may be immersive VR views having a 360 degree view.

FIG. 7 shows a conceptual illustration of a process that maps a remotespectator's seat selection to a real world capture system. Example seatselections 700 are shown to be mapped to various real world capturesystems represented by black dots in a seat mapping chart 716. Forexample, selection 702 for seat A8 is shown to map to real world capturesystem 718. Real world capture system 718 is shown to be mounted on topof the back rest of real world seat 720 corresponding to selection 702for seat A8. As a result, real world capture system 718 provides a viewwithin the real world space that is similar to a view a person wouldhave sitting in seat 720.

The real world capture system 718 is shown to include an array ofcameras 722 for capturing a 360 degree view of the real world, an arrayof microphones 724 for capturing sounds originating from variousdirections, and an array of sensors 726 for sensing environmental data.

The seat mapping chart 716 of FIG. 7 illustrates how seat selections 700are mapped to various real world capture systems. For rows A and B,there is a one-to-one mapping of a seat selection to a distinct realworld capture system. That is, every seat selection from rows A and Bmaps to its own real world capture system. For example, selection 704for seat A8 maps to a real world capture system that is specific to seatA8 and distinct from that of seat A9. For rows C and D, however, theseat mapping chart 716 shows that a group of four seat selections aremapped to one real world capture system. For example, a group 728 offour seats (e.g., C9, C10, D9, and D10) each map to one real worldcapture system. Thus, for example, a selection 706 for seat C9 may bemapped to real world capture system 730, as would a selection for seatsC10, D9, and D10. According to the embodiment shown, the real worldcapture system 730 is placed at a position that is in between or anaverage of the locations of each of seats C9, C10, D9, and D10. In otherembodiments, however, this need not be the case.

Seat selection 708 for seat E3 is shown to be mapped to real worldcapture system 734, which is associated with a group 736 of nine seats.The real world capture system 734 is thus shown to be mounted on top ofseat F9, which closest to an average of the locations of each of theseats in group 736.

The seat selection chart 716 shows groups of various sizes. For example,selection 710 is shown to map to a real world capture system that isassociated with a group 738 spanning 4 rows and 5 columns of seats.Moreover, both of selections 712 and 714 are shown to map to a realworld capture system associated with a group 740 having a 25 seatsarranged in a 5 by 5 pattern. While certain groupings of seats have beenshown in FIG. 7 for illustrative purposes, it is to be appreciated thatany size or pattern of groupings may be utilized with the methods andsystems presented here.

According to various embodiments, the real world capture systems 718,730, and 734 may be enabled to provide feedback signals to thereal-world audience such that the real-world audience is made aware thatone or more virtual spectators are connected. For example, each of thereal world capture system 718, 730, and 734 may include feedback systemsuch as one or more LEDs (not shown) for delivering feedback from thevirtual spectators. In some embodiments, the feedback system may includean array of LEDs that are configured to display a representation offeedback obtained from one or more virtual spectators for the real worldspectators. In these and other embodiments, the real world capturesystems 718, 730, and 734 may also include speakers (not shown) that areconfigured to deliver sounds from the virtual spectators to the realworld spectators. As a result, the virtual spectators are able toexpress their feedback to the real world space, which may reinforce theresponse and excitement of the real world arena.

In certain embodiments, the feedback system may be configured toidentify and convey the intensity and directionality of the virtualspectator feedback. As an example, if a virtual spectator begins toclap, his or her clapping may be represented by a dynamic pattern thatis displayed via an array of LEDs. The intensity of his or her clappingmay be represented by displaying various colors or patterns of movementwithin the array of LEDs. In some embodiments, more than one virtualspectator may be connected to a real world capture system at any givenmoment. In these embodiments, the intensity and pattern of feedback maybe correlated with the average reaction or response of the two or morevirtual spectators that are connected to the real world capture system.For example, if ten or more virtual spectators connected to the samereal world capture system provide an applause reaction, a pattern suchas a starburst may appear on a display mechanism of the real worldcapture system. As a result, the real world audience is provided withfeedback from the virtual spectators that is indicative of the number ofvirtual spectators that are connected to real world capture system inaddition to the nature of the response or reaction of the virtualspectators.

FIG. 8 shows a conceptual illustration of obtaining point of viewmapping data associated with the real world capture system from realworld space data and real world environment data, according to oneembodiment. Real world space data 800 is shown to include mapping dataof real world objects within the real world space. For example, realworld space data 800 may include coordinate locations of various realworld objects in the physical environment. In the embodiment shown, thereal world space may be the Bill Graham Civic Auditorium and the realworld space data may include the coordinate locations of the stage, thewalls, steps, walkways, the balcony level of the auditorium, as well asvarious points of interest for a given real world event held in theauditorium.

The real world space environment data 802 is shown to be captured by areal world capture system 801, which captures environmental data for thepoint of view of the seat or group of seats associated with the realworld capture device 801. As mentioned above, the real world capturedevice 801 may include an array of microphones, cameras, and sensors.

FIG. 8 shows that the real world space data 800 and the real worldenvironment data 802 may be combined to provide point of view mappingdata 804. Point of view mapping data 804 is shown to include real worldcoordinates, a distance to the stage (e.g., 20 meters), a distance toplayer 1 of team 1 (e.g., 25 meters), an acoustic reflectance of adirection θ, Φ, at w Hertz, a distance to a large rectangular ad, adistance to the commentators, a camera angle, a camera field of view, a3D map of the physical space, an acoustic map of the physical space, amap of audience members, a distance to the jumbotron, a distance to aspeaker, and/or a distance to a friend who is present either in personor virtually in the real world space.

According to some embodiments, the point of view mapping data 804 usesreal world space data 800 and real world space environment data 802 toconstruct camera coordinates of the real world and real world objects.For example, in some embodiments, point of view mapping data 804 is ableto transform “world coordinates” provided by the real world space datainto “camera coordinates” using real world environment data 802. As aresult, the point of view mapping data 804 is enabled to use the cameracoordinates of, for example, the large rectangular ad for reskinningpurposes, as the point of view mapping data 804 will know how the largerectangular ad projects onto the camera during image capture of the realworld space.

FIG. 9 shows a conceptual illustration of reskinning real world objectsin a real world view of an esports event. A first view 900 that has notbeen reskinned is shown to capture an esports event. The view 900, forexample, may be part of the video capture of a real world capturesystem. The first view 900 includes a plurality of advertisements placedon various surfaces that are viewable by the audience of the esportsevent. For example, first view 900 includes a left-wall ad 904 for milk,a right-wall ad for noodles, a first stage ad 908 for an energy drink, asecond stage ad 910 for a chipmaker, and a commentator desk ad 912 for aproduction company. The view also includes a main display 904 of thevideo game action that is being played, as well as team banners.

The first view 900 is shown to be inputted into reskinning logic 901,which in turn is shown to output a second view 900 of the esports eventhaving been reskinned. FIG. 9 demonstrates the graphical and visualcapabilities of the reskinning logic. For example, the left-wall ad 904′for milk has been reskinned to appear as left-wall ad 904′ forPlayStation. Likewise, the right-wall ad 906 for noodles and the secondstage ad 910 are each shown to have been reskinned to appear as aright-wall ad 906′ for Farpoint, and a second stage ad 910′ for adifferent chipmaker. For each of ads 904′, 906′, and ad 910′, thereskinning logic 901 has overlaid the replacement ads in a way thatoccupies the same space as the original ads 904, 906, and 910. Moreover,the reskinning logic 901 has performed the reskinning such that each ofthe replacement ads 904′ 906′ and 910′ are drawn to provide the sameperspective in the second view 902 as the respective original ads 904,906, and 910 provided in the first view 900. For example, left-wall ad904 appears in the first view 900 with a perspective having a skew. Inthe second view 902, the reskinning logic 901 overlays replacementleft-wall ad 904′ with a perspective having the same skew.

In addition to replacing or overlaying existing advertisements, thereskinning logic 901 is also able to detect additional real worldobjects such as portions of surfaces that may be reskinned with content.For example, back-wall ad 914 for a headphone manufacturer is shown tohave been reskinned in a region that is without content in the firstview 900. Likewise, stage-floor ad 916 is shown in the second view 902in a region where there was no existing content in the first view 900 ofthe real world space. the stage-floor ad 916 is further shown to havebeen reskinned to have geometric and perspectival consistency with thestage floor as captured by first view 900 by the real world capturedevice. That is, the stage-floor ad 916 appears as though it has beenlaid down flat on the stage floor.

As a result, the reskinning logic 901 contemplated here is able toidentify regions within the real world space as captured by the realworld capture device where content may be augmented where no contentexists in the real world space. The determination of whether and whichregions within the real world space that may be augmented withadditional content will depend upon the view received by each real worldcapture system. For example, a region within the real world space thatis determined to be desirable for augmenting content where no contentexists for a particular real world capture system may not be found to bedesirable by a different real world capture system because of adifference in points of view.

While in some embodiments, the replacement content is made to occupy thesame space as the original existing content, there are other instanceswhere the replacement content has a display dimension and aspect ratiothat is different than the existing content to be reskinned. Forexample, the first stage ad 908 for an energy drink is shown to havebeen reskinned with a replacement first stage ad 908′ for Star WarsBattlefront that has a wider dimension. In the embodiment shown,replacement first stage ad 908′ is appears in the second view 902 to beflush against the contours of the front of the stage instead ofappearing to float in air. As a result, the reskinning logic 901 is ableto reskin replacement content having different dimensions than thecontent to be replaced, while maintaining geometric and perspectivalconsistency with the real world object on which the replacement contentis overlaid.

Moreover, it is contemplated that additional regions may be reskinned.For example, in the first view 900, each of the player gaming stations918 is shown to display a live portrait of the respective player on agaming station audience-facing display. The live portrait displayprovides an audience with a way to see the facial expressions of theplayers that are otherwise blocked from view by respective computermonitors. As will be discussed in more detail below, embodimentscontemplated here enable the remote user to access these live portraitviews of players without having to rely on the gaming stationaudience-facing displays, which may be hard to make out. Thus, in someembodiments, reskinning may also be performed on the audience-facingdisplay of all or a portion of the gaming stations. For example, FIG. 9shows that each of the player gaming stations 918 first view 900 hasbeen reskinned by reskinning logic 901 to display letters reading“GEATNESS AWAITS” on the player gaming station 918′ in the second view902. Thus, it is contemplated that replacement content may be in theform of text in addition to graphical content such as ad graphics.

Under certain conditions, there may be original, existing ad contentthat may remain unchanged and not reskinned for a number or reasons. Inthe embodiment shown, the commentator desk ad 912 is shown to remainunchanged after the reskinning by the reskinning logic 901.

FIG. 10A shows a conceptual illustration of geometrically adjusting areplacement graphic 1002 for overlaying on top of a real world ad 1004for reskinning based on the perspective of the real world capture system1006. The real world capture system 1006 is shown to capture real worldad 1004 at a perspective that is not square or orthogonal to the planeof the real world ad 1004. It is contemplated that the real worldcapture system 1006 is enabled to determine distances (d: d₁, d₂, d₃,and d₄) to of the vertices, v₁, v₂, v₃, and v₄, of the real world ad1006, as well as the angles (θ: θ₁, θ₂, θ₃, and θ₄) formed between raystraced to each of the vertices. Based on d and θ, the reskinning logicis able to transform various graphical elements using a transformfunction to appear similar to the real world ad from the perspective ofthe real world capture system 1006.

The reskinning logic 1001 is shown to include a transform function Fthat is a function of d and θ, that maps image data from a set X toitself in either 2-dimensional space or 3-dimensional space. In theembodiment shown, replacement graphic 1002 is shown to be inputted intoreskinning logic, which maps pixel data associated with the replacementgraphic 1002 to new pixel coordinates in the modified replacementgraphic 1002. As a result, the modified replacement graphic 1002 will bemade to appear with the same perspective as the real world ad 1004 in aVR presentation.

FIG. 10B shows an embodiment of reskinning a real world object while thepoint of view of a real world capture device is changing. For example,the real world capture device is shown to travel longitudinally from apoint of view that is left of the real world object to a point of viewthat is to the right of the real world object. During its movement, thereskinning logic updates the parameters used for transforming andgeometrically an input graphic. As a result, the reskinning logic isable to output varying perspectives of the input graphic depending onthe current position and point of view of the real world capture device.

FIG. 11 shows additional real world objects that may be configurable tobe reskinned. In many esports events, there may be a multitude of videocameras that capture the players from different angles. In theembodiment shown, a shot 1000 of a player interacting with a video gameincludes a number of real world objects that may be reskinned. Forexample, regions 1102 and 1104 associate with the monitor the player isusing may be configurable to be reskinned. Certain other gaming gearlogos such as region 1106 associated with a “performance chair” andregion 1108 associated with head phones may also be regions that arecompatible with the reskinning logic described here. Moreover, graphics,logos, badges, patches, etc. worn on a player's attire may also becandidate real world objects configurable to be reskinned. For example,regions 1110 and 1112 associated with the player's team jersey or attireare also contemplated to be reskinned for a VR presentation.

FIG. 12 shows a conceptual illustration of a remote spectator 1201 beingpresented with an in-game fly-through view of the video game beingplayed at an esports event, according to one embodiment. The remotespectator receives a first view 1200 of the esports event that isprovided by a real world capture system located at the esports event.The remote spectator 1201 may wish to have an “fly-through” view (e.g.,first person view) of the game being played to view the game action froma point of view that is “inside” the game. Methods and systemscontemplated here allow a user to “jump in” to an esports video gamebeing played at an esports event to receive a fly-through view of thegame action. In the embodiment shown, the user points and drags afigurine to a position within the main display of the esports event. Inresponse, the VR presentation changes from a video stream provided bythe real world capture system to a second view 1202 that is afly-through view at a location corresponding to the position that thefigurine was placed. The fly-through view is rendered using live gamestate data of the esports game and may be performed on the HMD, at acomputing device associated with the HMD, or at a remote server.

FIG. 13 shows a conceptual illustration of a remote spectator 1301choosing between different views of an esports event for display,according to one embodiment. In the first view 1300, the remotespectator 1301 is provided with a video feed of the esports event from areal world capture system, as well as a plurality of viewing modules1304 for selection. The viewing modules may include various camera viewsof the real world such as portrait views of the players, views frombehind the players, views of the players interacting with a keyboard,mouse, touchpad, joystick, or other input device, views of individualplayer screens, fly-through views, views of the audience, views of thecommentators, etc. The remote spectator 1301 is shown to have selected aplayer 1 screen view 1306 and is in the process of selecting player 1front view 1308 by pointing and dragging 1310 the view module associatedwith the player 1 front view 1308 to a location within the VRpresentation that is desired for display.

In the second view 1302 of the VR presentation, the selected viewingmodules are shown to be displayed at the selected locations. Forexample, player 1 screen view 1306′ is shown to display a mirroring ofwhat player 1 sees on their own screen. Thus, the remote spectator 1301is able to see the control actions such as clicks and commands performedby player 1. Additionally, the player 1 front view 1308′ is shown todisplay a view of player 1 that may include facial expression, andcontroller input actions. As a result, a picture-in-picture orpictures-in-picture view of the esports event may be provided by themethods and system contemplated here, where a remote spectator 1301 isenabled to view desired views that they may not otherwise have access toas an in-person spectator. The ability to simultaneously view variouscamera views and game action views represent additional technologicaladvantages to current esports broadcasts, which do not give remotespectators the same degree of control of what they want to view.

FIG. 14 shows a diagram of an exemplary VR hosting server/computingdevice 1401 that enables VR presentation of an esports event via HMD toa user, according to one embodiment. The VR hosting server/computingdevice 1401 is shown to receive real world data from an Esports eventserver 1400. Depending on various embodiments, the esports event server1400 may be from a production company or a distribution company thatserves data captured at the esports event. The VR hostingserver/computing device 1401 is shown to obtain video streams 1404 thatinclude various camera views and game action views of the esports event,as well as space mapping data 1410 of the physical space or arenahosting the esports event, and space environment data 1412 captured bythe real world capture system. Additionally, VR hosting server/computingdevice 1401 is shown to receive a user profile 1406 that includes userpreferences and user data, as well as replacement content 1408, whichincludes, for example, replacement ads.

As mentioned earlier, the video streams 1404 may include a number ofvideo from a number of points of view. And even for a single point ofview, there may be multiple video streams that are used to create a 360degree (or less) field of view. In some embodiments, the multiple videostreams that are taken from one location for creating a 360 degree viewmay be stitched together by a remote server such as the Esports eventserver 1400 or at a production company or content distributer server. Inother embodiments, the video image stitching logic 1424 of the VRhosting server/computing device 1401 may perform a stitching of themultiple video streams having overlapping fields of view to render a 360degree view. The 360 degree view provided by video image stitching logic1424 is then fed to reskinning logic 1428. It should be appreciated thatalthough a 360 degree view may be recited by methods and systemspresented here, lesser than 360 degree views may be used as well, forexample, 180 degree views, or 150 degree views, and so on.

Also fed into reskinning logic 1428 are user preferences 1426 that areanalyzed from the user profile 1406. The user preferences 1426 helpdetermine which of replacement content (e.g., replacement ads) may be ofinterest to the remote spectator. It is to be appreciated that thereplacement content 1408 may also include video images for reskinningvideo clips into VR presentation 1440. Additionally, point of viewmapping data 1432 is also fed into the reskinning logic 1428 forperforming transforms such as geometric and perspectival modificationsto images or video provided by replacement content 1408.

Based on the video stream provided by the video image stitching logic1424, the user preferences 1426, the replacement content 1408, and thePOV mapping data 1432, the reskinning logic 1428 is enabled to reskinreal world objects within the video stream with the replacement contentin a way that maintains geometric and perspectival consistency with thereal world object used for reskinning. The reskinned video stream isthen fed to a VR graphics system 1430 that is configured to render a VRpresentation to the remote spectator via HMD interface 1438. Althoughnot shown, it is to be understood that various video compressionsprocesses and encoding may be performed at before, within, or aftervarious components shown in VR hosting server/computing device 1401. Forexample, a video feed outputted by VR graphics system 1430 may undergocompression before being transmitted to HMD interface 1438 for deliveryover a network to the HMD.

The HMD/Computing device 1401 is also shown to receive various soundstreams, including seat sound feed 1414 that includes sounds captured atthe real world capture system, background sound 1416 that includessounds and noises made by the crowd, commentator sound feed 1418,player/team sound feed 1420, and friends' voice feed 1422. It iscontemplated that method and systems described here enable the remotespectator to select between the various sound sources to be delivered inthe VR presentation 1440. The sound mixing logic 1434, depending on userselection, is configured to mix the various sounds sources to provide adesired sound output that is fed to a sound processing system 1426. Thesound processing system 1426 outputs an audio signal to the VRpresentation 1438 via the HMD interface 1438. Again, although not shown,various compression and decompression steps have been left out of theillustration for purposes of clarity. It will be understood, however,that various operations involving compression and decompression,encoding and decoding may be performed before, at, or after any of themodules or steps related to sound. For example, the sound outputprovided by sound processing system may be encoded before it istransmitted to the HMD via the HMD interface 1438.

FIG. 15 shows a conceptual illustration of two remote spectatorsinteracting with one another virtually within the context of an esportsevent, according to one embodiment. The two remote spectators are shownto have selected adjacent seats in the real world space. As a result,they may experience the esports event as if they were there together inperson via their respective VR presentations. In some embodiments, theVR presentation may render the VR scene to display a virtual presence ofthe other remote spectator and vice versa. Thus, each remote spectatorsees the other one. In some embodiments, such as the one shown, anavatar may be used for the virtual presence. In other embodiments, alive image or 3D representation of the one of the remote spectators maybe rendered in the VR presentation for the other remote spectator. It isalso envisioned that more than two spectators may participate in thegroup spectating experience described above.

In some embodiments, a remote spectator may be able to communicate witha live spectator who is also viewing the esports event. For example, ifreal world seats are equipped with a real world capture system includingcameras, microphones and speakers, a remote spectator may be able to seethe live spectator who is sitting next to them in the real world withinthe VR presentation.

FIG. 16 illustrates an additional embodiment of a HMD 1600 that may beused with the presented method and/or system. HMD 1600 includes hardwaresuch as processor 1602, battery 1604, facial recognition 1606, buttons,sensors, switches 1608, sound localization 1610, display 1612, andmemory 1614. HMD 1600 is also shown to include a position module 1626that comprises a magnetometer 1616, an accelerometer 1618, a gyroscope1620, a GPS 1622, and a compass 1624. Further included in robot 1600 arespeakers 1628, microphone 1630, LEDs 1632, object/s for visualrecognition 1634, IR lights 1636, front camera 1638, rear camera 1640,gaze tracking camera/s 1642, USB 1644, permanent storage 1646,vibro-tactile feedback 1648, communications link 1650, WiFi 1652,ultra-sonic communication 1654, Bluetooth 1656, near field communication(NFC) 1658, and radar 1660.

Although the method operations were described in a specific order, itshould be understood that other housekeeping operations may be performedin between operations, or operations may be adjusted so that they occurat slightly different times, or may be distributed in a system whichallows the occurrence of the processing operations at various intervalsassociated with the processing.

One or more embodiments can also be fabricated as computer readable codeon a computer readable medium. The computer readable medium is any datastorage device that can store data, which can be thereafter be read by acomputer system. Examples of the computer readable medium include harddrives, network attached storage (NAS), read-only memory, random-accessmemory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes and other optical andnon-optical data storage devices. The computer readable medium caninclude computer readable tangible medium distributed over anetwork-coupled computer system so that the computer readable code isstored and executed in a distributed fashion.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, it will be apparent thatcertain changes and modifications can be practiced within the scope ofthe appended claims. Accordingly, the present embodiments are to beconsidered as illustrative and not restrictive, and the embodiments arenot to be limited to the details given herein, but may be modifiedwithin the scope and equivalents of the appended claims.

What is claimed is:
 1. A method for delivering a virtual reality (VR)presentation of a real world space to a remote user via a head mounteddisplay (HMD), comprising: sending a request for the VR presentation ofthe real world space; identifying a seat selection within the real worldspace made by the user, the seat selection being mapped to a real worldcapture system in the real world space, the real world capture systemconfigured for capturing video and audio at a location that correspondsto the seat selection; receiving mapping data for the real world capturesystem, the mapping data including real world coordinates for the realworld capture system; accessing a user profile having user preferencesof the user; receiving a video stream of the real world space from thereal world capture system, the video stream including a plurality ofvideo images captured by one or more cameras of the real world capturesystem, the plurality of images being presented in the HMD from aperspective associated with the real world coordinates of the real worldcapture system corresponding to the seat selection; identifying, fromthe plurality of images of the video stream, a real world objectconfigured for reskinning during presentation to the user; reskinning,based on the user profile, the real world object within the plurality ofvideo images, the reskinning includes overlaying a graphical contentelement in place of the image data associated with the real worldobject, the overlaying being geometrically adjusted based on theperspective associated with the real world capture system correspondingto the seat selection; and presenting the video stream to the user viathe HMD for the VR presentation.
 2. The method of claim 1, furthercomprising: tracking, from a plurality of sensors, a position andorientation of the HMD while the user is wearing the HMD; and detecting,based on the tracking the position and orientation of the HMD, amovement of the HMD while the user is wearing the HMD, wherein a fieldof view of the video stream is made to change in response to themovement of the HMD.
 3. The method of claim 1, wherein the video streamprovided by the real world capture system includes a 360 horizontaldegree field of view.
 4. The method claim 1, wherein the mapping datafurther includes coordinates of the real world object, the coordinatesof the real world object usable for the reskinning the real worldobject.
 5. The method of claim 1, where the user preferences include ademographic of the user, a browsing history of the user, a shoppinghistory of the user, a viewing history of the user, social media data ofthe user, or user interest, the user preferences usable to determine thegraphical content element for the reskinning.
 6. The method of claim 1,wherein the overlaying includes measuring real world distances betweenthe real world capture system and vertices of the real world object, andmeasuring angles formed between rays that are traced from the real worldcapture system to the vertices, wherein the distances measured andangles measured are used for the overlaying the graphical content itembeing geometrically adjusted based on the perspective of the real worldcapture system.
 7. The method of claim 6, wherein the real world objectmoves and wherein the measuring the real world distances and themeasuring the angles occur in real-time or near real-time as the realworld object moves for such that the graphical content item isgeometrically adjusted based on a changing perspective of real worldobject as it moves.
 8. The method of claim 1, wherein the reskinningincludes identifying vertices of the real world object within theplurality of video images and transforming the graphical content elementsuch that vertices of the graphical content item fit to the vertices ofthe real world object, the graphical content element having beentransformed being used for the overlaying the graphical content elementin place of the image data associated with the real world object.
 9. Themethod of claim 1, further comprising: receiving an audio feed of thereal world space from the real world capture system, the audio feedbeing processed according to real world coordinates of the real worldcapture system associated with the seat location.
 10. The method ofclaim 1, further comprising: receiving a selection of a video modulefrom the user, the video module being an additional video stream of thereal world space, wherein the video module is displayed as apicture-in-picture display during said presenting the video stream tothe user via the HMD.
 11. The method of claim 9, wherein the real worldspace includes electronic sports (esports) activity, the video module isone of an additional video stream of a front perspective of a player, arear perspective of a player, a screen of player, a perspective of acontrol pad of a player, a perspective of a jumbotron of a game, aperspective of one or more commentators, a perspective of a team ofplayers, or a perspective of an audience member.
 12. The method of claim1, further comprising: receiving a command from the user for afly-through view of a video game being played in the real world space,the command including a location within the video game for thefly-through view; receiving, from the real world space, a stream of agame state of the video game being played; rendering, based on thestream of the game state and on a virtual camera view associated withthe location within the video game for the fly-through view, thefly-through view; and presenting the fly-through view to the user viathe HMD.
 13. A server system, comprising: one or more processors; and amemory disposed in communication with the one or more processors andstoring processor-executable instructions to: send a request for a VRpresentation of a real world space, the request being for delivering theVR presentation of the real world space to a user via an HMD; identify aseat selection within the real world space made by the user, the seatselection being mapped to a real world capture system in the real worldspace, the real world capture system configured for capturing video andaudio at a location that corresponds to the seat selection; receivemapping data for the real world capture system, the mapping dataincluding real world coordinates for the real world capture system;receive a video stream of the real world space from the real worldcapture system, the video stream including a plurality of video imagescapture by one or more cameras of the real world capture system, theplurality of images being presented in the HMD from a perspectiveassociated with the real world coordinates of the real world capturesystem corresponding to the seat selection; identify, from the pluralityof images of the video stream, a real world object configured forreskinning during presentation to the user; reskin, based on the userprofile, the real world object within the plurality of video images, thereskinning includes overlaying a graphical content element in place ofthe image data associated with the real world object, the overlayingbeing geometrically adjusted based on the perspective associated withthe real world capture system corresponding to the seat selection; andpresent the video stream to the user via the HMD for the VRpresentation.
 14. The server system of claim 13, further comprisinginstructions to: track, from a plurality of sensors, a position andorientation of the HMD while the user is wearing the HMD; and detect,based on the tracking the position and orientation of the HMD, amovement of the HMD while the user is wearing the HMD, wherein a fieldof view of the video stream is made to change in response to themovement of the HMD.
 15. The server system of claim 13, wherein themapping data further includes coordinates of the real world object, thecoordinates of the real world object usable for the reskinning the realworld object.
 16. The server system of claim 13, wherein the overlayingincludes measuring real world distances between the real world capturesystem and vertices of the real world object and measuring angles formedbetween rays that are traced from the real world capture system to thevertices, wherein the distances measured and the angles measured areused for the overlaying the graphical content item being geometricallyadjusted based on the perspective of the real world capture system. 17.The server system of claim 13, wherein the real world object moves andwherein the measuring the real world distances and the measuring theangles occur in real-time or near real-time as the real world objectmoves for such that the graphical content item is geometrically adjustedbased on a changing perspective of real world object as it moves. 18.The server system of claim 13, wherein the reskinning includesidentifying vertices of the real world object within the plurality ofvideo images and transforming the graphical content element such thatvertices of the graphical content item fit to the vertices of the realworld object, the graphical content element having been transformedbeing used for the overlaying the graphical content element in place ofthe image data associated with the real world object.
 19. Anon-transitory medium storing processor-executable instructions, theinstructions comprising instructions to: send a request for a VRpresentation of a real world space, the request being for delivering theVR presentation of the real world space to a user via an HMD; identify aseat selection within the real world space made by the user, the seatselection being mapped to a real world capture system in the real worldspace, the real world capture system configured for capturing video andaudio at a location that corresponds to the seat selection; receivemapping data for the real world capture system, the mapping dataincluding real world coordinates for the real world capture system;receive a video stream of the real world space from the real worldcapture system, the video stream including a plurality of video imagescapture by one or more cameras of the real world capture system, theplurality of images being presented in the HMD from a perspectiveassociated with the real world coordinates of the real world capturesystem corresponding to the seat selection; identify, from the pluralityof images of the video stream, a real world object configured forreskinning during presentation to the user; reskin, based on the userprofile, the real world object within the plurality of video images, thereskinning includes overlaying a graphical content element in place ofthe image data associated with the real world object, the overlayingbeing geometrically adjusted based on the perspective associated withthe real world capture system corresponding to the seat selection; andpresent the video stream to the user via the HMD for the VRpresentation.
 20. The non-transitory medium of claim 19, wherein theoverlaying includes measuring real world distances between the realworld capture system and vertices of the real world object and measuringangles formed between rays that are traced from the real world capturesystem to the vertices, wherein the distances measured and the anglesmeasured are used for the overlaying the graphical content item beinggeometrically adjusted based on the perspective of the real worldcapture system.